Low Complexity Multiview Video Coding

3D video is a technology that has seen a tremendous attention in the recent years. Multiview Video Coding (MVC) is an extension of the popular H.264 video coding standard and is commonly used to compress 3D videos. It offers an improvement of 20% to 50% in compression efficiency over simulcast encoding of multiview videos using the conventional H.264 video coding standard. However, there are two important problems associated with it: (i) its superior compression performance comes at the cost of significantly higher computational complexity which hampers the real-world realization of MVC encoder in applications such as 3D live broadcasting and interactive Free Viewpoint Television (FTV), and (ii) compressed 3D videos can suffer from packet loss during transmission, which can degrade the viewing quality of the 3D video at the decoder. This thesis aims to solve these problems by presenting techniques to reduce the computational complexity of the MVC encoder and by proposing a consistent error concealment technique for frame losses in 3D video transmission. The thesis first analyses the complexity of the MVC encoder. It then proposes two novel techniques to reduce the complexity of motion and disparity estimation. The first method achieves complexity reduction in the disparity estimation process by exploiting the relationship between temporal levels, type of macroblocks and search ranges while the second method achieves it by exploiting the geometrical relation- ship between motion and disparity vectors in stereo frames. These two methods are then combined with other state-of-the-art methods in a unique framework where gains add up. Experimental results show that the proposed low-complexity framework can reduce the encoding time of the standard MVC encoder by over 93% while maintaining similar compression efficiency performance. The addition of new View Synthesis Prediction (VSP) modes to the MVC encoding framework improves the compression efficiency of MVC. However, testing additional modes comes at the cost of increased encoding complexity. In order to reduce the encoding complexity, the thesis, next, proposes a bayesian early mode decision technique for a VSP enhanced MVC coder. It exploits the statistical similarities between the RD costs of the VSP SKIP mode in neighbouring views to terminate the mode decision process early. Results indicate that the proposed technique can reduce the encoding time of the enhanced MVC coder by over 33% at similar compression efficiency levels. Finally, compressed 3D videos are usually required to be broadcast to a large number of users where transmission errors can lead to frame losses which can degrade the video quality at the decoder. A simple reconstruction of the lost frames can lead to inconsistent reconstruction of the 3D scene which may negatively affect the viewing experience of a user. In order to solve this problem, the thesis proposes, at the end, a consistency model for recovering frames lost during transmission. The proposed consistency model is used to evaluate inter-view and temporal consistencies while selecting candidate blocks for concealment. Experimental results show that the proposed technique is able to recover the lost frames with high consistency and better quality than two standard error concealment methods and a baseline technique based on the boundary matching algorithm

Low-Complexity Multiview Video Coding

We consider the problem of complexity reduction in Multiview Video Coding (MVC). We provide a unique comprehensive study that integrates and compares the different low complexity encoding techniques that have been proposed at different levels of the MVC system. In addition, we propose a novel complexity reduction method that takes advantage of the relationship between disparity vectors along time. The relationship is exploited with respect to the motion activity in the frame, as well as with the position of the frame in the Group of Pictures. We integrate this technique into our unique comprehensive framework and evaluate the performance of the resulting system in different setups. We show that the effective combination of complexity reduction techniques results in saving up to 93% in encoding time at the cost of only 0.08 dB in peak signal-to-noise ratio (PSNR) and 1.64% increase in bitrate compared to the standard MVC implementation (JMVM 6.0)

Sensor Fusion for Identification of Freezing of Gait Episodes Using Wi-Fi and Radar Imaging

Parkinson’s disease (PD) is a progressive and neurodegenerative condition causing motor impairments. One of the major motor related impairments that present biggest challenge is freezing of gait (FOG) in Parkinson’s patients. In FOG episode, the patient is unable to initiate, control or sustain a gait that consequently affects the Activities of Daily Livings (ADLs) and increases the occurrence of critical events such as falls. This paper presents continuous monitoring ADLs and classification freezing of gait episodes using Wi-Fi and radar imaging. The idea is to exploit the multi-resolution scalograms generated by channel state information (CSI) imprint and micro-Doppler signatures produced by reflected radar signal. A total of 120 volunteers took part in experimental campaign and were asked to perform different activities including walking fast, walking slow, voluntary stop, sitting down & stand up and freezing of gait. Two neural networks namely Autoencoder and a proposed enhanced Autoencoder were used classify ADLs and FOG episodes using data fusion process by combining the images acquired from both sensing techniques. The Autoencoder provided overall classification accuracy of ~87% for combined datasets. The proposed algorithm provided significantly better results by presenting an overall accuracy of ~98% using data fusion

Fast encoding techniques for Multiview Video Coding

Multiview Video Coding (MVC) is a technique that permits efficient compression of multiview video. MVC uses variable block size motion and disparity estimation for block matching. This requires an exhaustive search process that involves all possible macro-block partition sizes. We analyze the time complexity of MVC and the methods that have been proposed to speed up motion and disparity estimation. We then propose two new methods: Previous Disparity Vector Disparity Estimation (PDV-DE) and Stereo-Motion Consistency Constraint Motion and Disparity Estimation (SMCC-MDE). PDV-DE exploits the correlation between temporal levels and disparity vectors to speed up the disparity estimation process while SMCC-MDE exploits the geometrical relationship of consecutive frame pairs to speed up motion and disparity estimation. We build a complete low complexity MVC encoding solution that combines our two methods with complementary previous methods to speed up motion and disparity search. We evaluate the complexity of our solution in terms of encoding time and number of search points. Our experimental results show that our solution can reduce the encoding time and number of search points of the standard MVC implementation (JMVM 6.0) using the fast TZ search mode up to 93.7% and 96.9%, respectively, with negligible degradation in the rate-distortion performance. Compared to the best published results, this is an improvement of up to 11% and 7%, respectively. (C) 2013 Elsevier B.V. All rights reserved

Bayesian Early Mode Decision Technique for View Synthesis Prediction-Enhanced Multiview Video Coding

View synthesis prediction (VSP) is a codingmode that predicts video blocks from synthesised frames. It is particularly useful in a multi-camera setup with large inter-camera distances. Adding a VSP-based SKIP mode to a standard Multiview Video Coding (MVC) framework improves the rate-distortion (RD) performance but increases the time complexity of the encoder. This letter proposes an earlymode decision technique for VSP SKIP-enhanced MVC. Our method uses the correlation between the RD costs of the VSP SKIP mode in neighbouring views and Bayesian decision theory to reduce the number of candidate coding modes for a given macroblock. Simulation results showed that our technique can save up to 36.20% of the encoding time without any significant loss in RD performance

Temporal and Inter-view Consistent Error Concealment Technique for Multiview plus Depth Video

Multiview plus depth (MVD) is an emerging video format with many applications, including 3D television and free viewpoint television. During broadcast of compressed MVD video, transmission errors may cause the loss of whole frames, resulting in significant degradation of video quality. Error concealment techniques have been widely used to deal with transmission errors in video communication. However, the existing solutions do not address the requirement that the reconstructed frames be consistent with neighbouring frames, i.e., corresponding pixels have consistent color information. We propose a new consistency model for error concealment of MVD video that allows to maintain a high level of consistency between frames of the same view (temporal consistency) and those of neighbouring views (inter-view consistency). We then propose an algorithm that uses our model to implement concealment in a consistent way. Simulations with the reference software for the Multiview Video Coding project of the Joint Video Team (JVT) of the ISO/IEC MPEG and ITU-T VCEG show that our method outperforms benchmark techniques, including a baseline approach based on the Boundary Matching Algorithm, with respect to both reconstruction quality and view consistency

Early CU Depth Decision and Reference Picture Selection for Low Complexity MV-HEVC

The Multi-View extension of High Efficiency Video Coding (MV-HEVC) has improved the coding efficiency of multi-view videos, but this comes at the cost of the extra coding complexity of the MV-HEVC encoder. This coding complexity can be reduced by efficiently reducing time-consuming encoding operations. In this work, we propose two methods to reduce the encoder complexity. The first one is Early Coding unit Splitting (ECS), and the second is the Efficient Reference Picture Selection (ERPS) method. In the ECS method, the decision of Coding Unit (CU) splitting for dependent views is made on the CU splitting information obtained from the base view, while the ERPS method for dependent views is based on selecting reference pictures on the basis of the temporal location of the picture being encoded. Simulation results reveal that our proposed methods approximately reduce the encoding time by 58% when compared with HTM (16.2), the reference encoder for MV-HEVC

Isolation design flow effectiveness evaluation methodology for Zynq SoCs

Static Random-Access Memory (SRAM)-based Field Programmable Gate Arrays (FPGAs) are increasingly being used in many application domains due to their higher logic density and reconfiguration capabilities. However, with state-of-the-art FPGAs being manufactured in the latest technology nodes, reliability is becoming an important issue, particularly for safety-critical avionics, automotive, aerospace, industrial robotics, medical, and financial systems. Therefore, fault tolerant system design methodologies have become essential in the aforementioned application domains. The Isolation Design Flow (IDF) is one such design methodology that has promising prospects due to its ability to isolate logic design modules at the physical level for fault containment purposes. This paper proposes a methodology to evaluate the effectiveness of the IDF. To do so, reverse engineering is used to enable fault injection on the IDF designs with minimal changes in the bit-stream. This reduces the time needed to inject a fault significantly thus accelerating the evaluation process. Then this methodology is applied to a case study of a single-chip cryptography application on a ZynQ SoC. Specifically, an Advanced Encryption Standard (AES) Duplication With Comparison (DWC) design is physically isolated with IDF and subsequently subjected to frame-level Fault Injection (FI) in the configuration memory